Authentica AI: An Explainable Multimodal System for Synthetic Media Detection Using Deep Features, Reconstruction Error and Frequency-Domain Forensics

Abstract

The rapid improvement of generative models has made AI-generated images, cloned voices and manipulated video increasingly difficult to distinguish from authentic media, while most deployed detectors examine a single modality and return an opaque score. This paper presents Authentica AI, a web-based multimodal synthetic-media detection system that automatically routes each upload to a modality-specific detector. Images are analysed by a pretrained convolutional backbone (EfficientNet-B0, with a ResNet-50 variant), a diffusion-reconstruction error map, and five handcrafted forensic signals — spectral energy loss, spatial variation, edge-sharpness consistency, colour-channel correlation and local-variance entropy — which are fused by a weighted ensemble and a calibrated sigmoid into a single synthetic-probability score. Audio is converted to log-Mel spectrograms and classified by a ResNet-18 model. Every image verdict is accompanied by a forensic heatmap that localises the regions that drove the decision, and results are stored in a PostgreSQL database and presented through an interactive dashboard with batch processing. In a prototype evaluation on a five-image batch the system returned verdicts with a mean latency of 263.3 ms per image on a CPU, and per-stage profiling of a single run (288.1 ms) shows that the reconstruction stages account for about two thirds of the processing time. The paper also states the limits of this evaluation: no labelled benchmark accuracy is reported, and large-scale benchmarking and video support are identified as future work.

Introduction

Authentica AI is an artificial intelligence-based platform designed to identify AI-generated or manipulated digital media, commonly known as deepfakes. With the advancement of Generative Adversarial Networks (GANs) and diffusion models, synthetic images, cloned voices, and manipulated videos have become increasingly realistic. These developments create challenges for detecting misinformation, impersonation, and digitally manipulated content.

The proposed system aims to address the limitations of existing deepfake detectors, including their dependence on a single media type, reduced performance on unfamiliar AI generators or compressed files, and lack of clear explanations for their predictions.

Authentica AI uses a media-type router to identify uploaded files and direct them to the appropriate detection pipeline. In its current implementation, it supports image formats such as JPG, PNG, and WebP, along with audio formats such as WAV, MP3, and M4A. Video detection is planned for future development.

The image detection pipeline combines a deep-learning model, diffusion-reconstruction error analysis, and five handcrafted forensic signals related to frequency distribution, texture, edge sharpness, colour-channel correlation, and local variance. These features are combined using a weighted ensemble to produce a synthetic-probability score. A forensic heatmap is also generated to highlight image regions that may contain evidence of manipulation.

For audio detection, the system uses FFmpeg for audio processing, converts audio into log-Mel spectrograms, and applies a ResNet-18 classifier to identify patterns associated with AI-generated speech and voice cloning. The resulting spectrogram is displayed alongside the detection result.

The platform includes a web interface with a confidence gauge, performance metrics, batch analysis, processing-stage information, and persistent result storage using PostgreSQL. It provides users with an authenticity verdict, confidence score, and supporting visual evidence to help them understand the prediction.

The literature review examines multimodal deepfake detection, image forensics, frequency-domain analysis, diffusion-reconstruction methods, and explainable AI. It highlights the need for a unified detection system that combines multiple forensic techniques with interpretable results.

Conclusion

Authentica AI shows that a modality-aware router, a hybrid image pipeline that fuses deep features with reconstruction error and frequency-domain forensics, and an explainable heatmap interface can be delivered as a single web platform that runs at interactive speed on a CPU. Its per-stage profile identifies the reconstruction pass as the main cost, and its limitations are stated above. Future work will: (1) evaluate the system on public labelled benchmarks and report accuracy, AUC and robustness to compression and noise; (2) extend detection to video using a Vision Transformer [6] frame analyser for face-swap, lip-sync and temporal inconsistencies; (3) add Grad-CAM [12] explanations alongside the forensic heatmap; (4) train and fine-tune on newer generators such as Stable Diffusion 3, Midjourney v6 and Sora; (5) add cross-modal consistency analysis so that image, audio and video from the same source can be evaluated jointly; and (6) provide a rate-limited, monitored REST API for production use.

References

[1] M. A. Raza and K. M. Malik, “Multimodaltrace: Deepfake detection using audiovisual representation learning,” IEEE Xplore, 2023. [2] L. Lin, N. Gupta, Y. Zhang, H. Ren, C.-H. Liu, F. Ding, X. Wang, X. Li, L. Verdoliva, and S. Hu, “Detecting multimedia generated by large AI models: A survey,” arXiv preprint arXiv:2402.00045, 2024. [3] K. Lee, Y. Zhang, and Z. Duan, “A multi-stream fusion approach with one-class learning for audio-visual deepfake detection,” in Proc. IEEE Int. Workshop on Multimedia Signal Processing (MMSP), 2024, pp. 1–6. [4] M. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proc. 36th Int. Conf. on Machine Learning (ICML), 2019. [5] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016. [6] A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Int. Conf. on Learning Representations (ICLR), 2021. [7] Z. Wang, J. Bao, W. Zhou, W. Wang, H. Hu, H. Chen, and H. Li, “DIRE for diffusion-generated image detection,” in Proc. IEEE/CVF Int. Conf. on Computer Vision (ICCV), 2023. [8] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems (NeurIPS), 2020. [9] I. Goodfellow et al., “Generative adversarial nets,” in Advances in Neural Information Processing Systems (NeurIPS), 2014. [10] R. Durall, M. Keuper, and J. Keuper, “Watch your up-convolution: CNN based generative deep neural networks are failing to reproduce spectral distributions,” in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2020. [11] J. Frank, T. Eisenhofer, L. Schönherr, A. Fischer, D. Kolossa, and T. Holz, “Leveraging frequency analysis for deep fake image recognition,” in Proc. 37th Int. Conf. on Machine Learning (ICML), 2020. [12] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” in Proc. IEEE Int. Conf. on Computer Vision (ICCV), 2017. [13] S.-Y. Wang, O. Wang, R. Zhang, A. Owens, and A. A. Efros, “CNN-generated images are surprisingly easy to spot… for now,” in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.

Copyright

Copyright © 2026 Amaresh Kannan K, Aravindh NK, Mrs. Sindhiya R. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.